---
title: Integration Apps
description: "Integration apps allow OMI to interact with external services and process data in real-time. This guide will walk you through creating both Memory Creation Triggers and Real-Time Transcript Processors."
---

## Types of Integration Apps

### 1. 👷 Memory Creation Triggers

#### Video Walkthrough

<iframe
  width="560"
  height="315"
  src="https://www.youtube.com/embed/OOjWeGhuLeY?si=6Ya1GwqL4GSlq3mi"
  title="YouTube video player"
  frameborder="0"
  allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share"
  referrerpolicy="strict-origin-when-cross-origin"
  allowfullscreen
></iframe>

**Running a FastAPI app locally, not on AWS:**

<iframe
  width="560"
  height="315"
  src="https://www.youtube.com/embed/bMU6fTLysRY?si=3cvXEsWAUwKEnjHn"
  title="YouTube video player"
  frameborder="0"
  allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share"
  referrerpolicy="strict-origin-when-cross-origin"
  allowfullscreen
></iframe>

These apps are activated when OMI creates a new memory, allowing you to process or store the memory data
externally.

[![Memory trigger app](https://img.youtube.com/vi/Yv7gP3GZ0ME/0.jpg)](https://youtube.com/shorts/Yv7gP3GZ0ME)

#### Example Use Cases

- Update project management tools with conversation summaries
- Create a personalized social platform based on conversations and interests
- Generate a knowledge graph of interests, experiences, and relationships

### 2. 🏎️ Real-Time Transcript Processors

#### Video Walkthrough:

<iframe
  width="560"
  height="315"
  src="https://www.youtube.com/embed/86D4v3n1o48?si=tQl-s9jikupjix0c"
  title="YouTube video player"
  frameborder="0"
  allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share"
  referrerpolicy="strict-origin-when-cross-origin"
  allowfullscreen
></iframe>

These apps process conversation transcripts as they occur, enabling real-time analysis and actions.

[![Memory trigger app](https://img.youtube.com/vi/h4ojO3WzkxQ/0.jpg)](https://youtube.com/shorts/h4ojO3WzkxQ)

#### Example Use Cases

- Live conversation coaching and feedback
- Real-time web searches or fact-checking
- Emotional state analysis and supportive responses

### 3. 🔊 Real-Time Audio Bytes (audio_bytes)

#### Guide:

Omi can stream raw audio bytes from a DevKit directly to an external endpoint. This integration uses the trigger type `audio_bytes` and is useful for applications that need raw PCM audio (VAD, custom ASR, feature extraction, or long-term storage).

Key points:

- Trigger name: `audio_bytes`
- Request: POST
- Query parameters appended by the backend: `sample_rate` (e.g. 16000) and `uid`
- Content-Type: `application/octet-stream` (raw PCM16 bytes)

Example request:

`POST /your-endpoint?sample_rate=16000&uid=user123` with the request body containing raw PCM16 (16-bit little-endian) audio bytes.

Notes and recommendations:

- The bytes are raw PCM16 (16-bit little-endian), 2 bytes per sample. To produce a playable WAV file you must prepend a WAV header and concatenate the received chunks.
- You control delivery frequency in the Omi app Developer Settings (the webhook field accepts `url,seconds` — the second value configures how many seconds of audio are sent per request).
- For an example implementation and deployment-ready code, see the Real-Time Audio Streaming guide: `doc/developer/apps/AudioStreaming.mdx`.

#### Example Use Cases

- Voice activity detection (VAD) and endpointing
- Feed custom ASR or speech models for research or improved accuracy
- Extract audio features (spectrograms, embeddings) in real-time
- Store raw audio chunks to cloud storage for later processing

## Creating an Integration App

### Step 1: Define Your App 🎯

Decide whether you're creating a Memory Creation Trigger or a Real-Time Transcript Processor, and outline its specific
purpose.

### Step 2: Set Up Your Endpoint 🔗

Create an endpoint (webhook) that can receive and process the data sent by OMI. You can [create a test webhook](https://webhook-test.com/). The data structure will differ based on your
app type:

#### For Memory Creation Triggers:

Your endpoint will receive the entire memory object as a JSON payload, with a `uid` as a query parameter. Here is what to
expect:

`POST /your-endpoint?uid=user123`

```json
{
  "id": 0,
  "created_at": "2024-07-22T23:59:45.910559+00:00",
  "started_at": "2024-07-21T22:34:43.384323+00:00",
  "finished_at": "2024-07-21T22:35:43.384323+00:00",
  "transcript_segments": [
    {
      "text": "Segment text",
      "speaker": "SPEAKER_00",
      "speakerId": 0,
      "is_user": false,
      "start": 10.0,
      "end": 20.0
    }
    // More segments...
  ],
  "photos": [],
  "structured": {
    "title": "Conversation Title",
    "overview": "Brief overview...",
    "emoji": "🗣️",
    "category": "personal",
    "action_items": [
      {
        "description": "Action item description",
        "completed": false
      }
    ],
    "events": []
  },
  "apps_response": [
    {
      "app_id": "app-id",
      "content": "App response content"
    }
  ],
  "discarded": false
}
```

Your app should process this entire object and perform any necessary actions based on the full context of the memory.

> Check the [Notion CRM Python Example](https://github.com/BasedHardware/Omi/blob/bab12a678f3cfe43ab1a7aba62645222de4378fb/apps/example/main.py#L85)
> and it's respective JSON format [here](https://github.com/BasedHardware/Omi/blob/bab12a678f3cfe43ab1a7aba62645222de4378fb/community-plugins.json#L359).

**For Real-Time Transcript Processors:**

Your endpoint will receive a JSON payload containing the most recently transcribed segments, with both session_id and
uid as query parameters. Here's the structure:

`POST /your-endpoint?session_id=abc123&uid=user123`

```json
[
  {
    "text": "Segment text",
    "speaker": "SPEAKER_00",
    "speakerId": 0,
    "is_user": false,
    "start": 10.0,
    "end": 20.0
  }
  // More recent segments...
]
```

**Key points for Real-Time Transcript Processors:**

1. Segments arrive in multiple calls as the conversation unfolds.
2. Use the session_id to maintain context across calls.
3. Implement smart logic to avoid redundant processing.
4. Consider building a complete conversation context by accumulating segments.
5. Clear processed segments to prevent re-triggering on future calls.

Remember to handle errors gracefully and consider performance, especially for lengthy conversations!

> Check the Realtime News checker [Python Example](https://github.com/BasedHardware/omi/blob/bab12a678f3cfe43ab1a7aba62645222de4378fb/plugins/example/main.py#L100)
> and it's respective JSON format [here](https://github.com/BasedHardware/Omi/blob/bab12a678f3cfe43ab1a7aba62645222de4378fb/community-plugins.json#L379).

### Step 3: Test Your App 🧪

Time to put your app through its paces! Follow these steps to test both types of integrations:

1. Open the OMI app on your device.
2. Go to Settings and enable Developer Mode.
3. Navigate to Developer Settings.

#### For Memory Creation Triggers:

4. Set your endpoint URL in the "Memory Creation Webhook" field. If you don't have an endpoint yet, [create a test webhook](https://webhook-test.com/)
5. To test without creating a new memory:
   - Go to any memory detail view.
   - Click on the top right corner (3 dots menu).
   - In the Developer Tools section, trigger the endpoint call with existing memory data.

[![Memory trigger app](https://img.youtube.com/vi/dYVSbEpoV0U/0.jpg)](https://youtube.com/shorts/dYVSbEpoV0U)

#### For Real-Time Transcript Processors:

4. Set your endpoint URL in the "Real-Time Transcript Webhook" field.
5. Start speaking to your device - your endpoint will receive real-time updates as you speak.

[![Memory trigger app](https://img.youtube.com/vi/CHz9JnOGlTQ/0.jpg)](https://youtube.com/shorts/CHz9JnOGlTQ)

Your endpoints are now ready to spring into action!

For **Memory Creation Triggers**, you can test with existing memories or wait for new ones to be created.

For **Real-Time Processors**, simply start a conversation with OMI to see your app in action.

Happy app crafting! We can't wait to see what you create! 🎉

### Step 4: Submit Your App

Submit your app using the Omi mobile app.

The **webhook URL** should be a POST request in which the memory object is sent as a JSON payload.

The **setup completed URL** is optional and should be a GET endpoint that returns `{'is_setup_completed': boolean}`.

The **auth URL** is optional as well and is utilized by users to setup your app. The `uid` query paramater will be appended to this URL upon usage.

The setup instructions can be either a link to instructions or text instructions for users on how to setup your app.

### Setup Instructions Documentation

When writing your setup instructions, consider including:

1. A step-by-step setup guide
2. Screenshots (if applicable)
3. Authentication steps (if required)
4. Troubleshooting tips

Example structure:

**Notes:**

- Authentication is not needed for all apps. Include only if your app requires user-specific setup or credentials.
- For apps without authentication, users can simply enable the app without additional steps.
- All your README links, when the user opens them, we'll append a `uid` query parameter to it, which you can use to
  associate setup or credentials with specific users.
